1. Field of the Invention
The present invention relates to an image processing apparatus, and particularly relates to a technology for measuring depth information on a subject from a plurality of images taken under different imaging parameters.
2. Description of the Related Art
Conventionally, as a method for acquiring the depth of an imaging scene from an image acquired by an image pickup apparatus, there is proposed a depth from defocus (DFD) method as disclosed in Japanese Patent Application Laid-open No. H01-167610. In DFD method, a plurality of images having different blurs are acquired by controlling imaging parameters of an image pickup optical system, and the correlation amount of the mutual blurs is calculated by using a measurement target pixel and its surrounding pixel in the plurality of the acquired images.
The correlation amount of the blur changes according to the depth of a subject in the image, and hence, by using the relationship, the depth is calculated from the correlation amount of the blur. Depth measurement by DFD method allows calculation of the depth by using one image pickup system so that the depth measurement thereby has an advantage that it can be incorporated in a commercial image pickup apparatus.
The depth estimation method by DFD method described in Japanese Patent Application Laid-open No. H01-167610 calculates the depth from the blur of the taken image. The size of the blur by the image pickup optical system basically changes according to the depth to the subject, and also changes according to the spatial frequency of the subject. Consequently, depth accuracy is changed according to the spatial frequency in the image. In addition, there has been a problem that, in a case where a plurality of the spatial frequencies are present in an image region used for depth measurement, depth estimation accuracy is especially reduced.
In Japanese Patent Application Laid-open No. 2001-208524, in order to cope with the case where a plurality of the spatial frequencies are present, frequency processing is performed by using different Gabor filters in three directions at three central frequencies and the depth is then calculated at each frequency. Further, threshold processing is performed on the calculated depth and a label image is generated. Then, in a local region around a depth measurement pixel, when all labels calculated by using a lower frequency are identical with each other, the label of the depth measurement pixel in the label image calculated by using a higher frequency is replaced with the label of the lower frequency.
Japanese Patent Application Laid-open No. 2001-208524 proposes a region segmentation method of the image using the depth by the method mentioned above. However, there is a problem that variations occur in depth calculation depending on the frequency of the subject due to the extraction of a frequency component by a predetermined band-pass filter.
In addition, since filter processing to a plurality of spatial frequencies and a plurality of directions is performed, there are problems such as an increase in the number of times of filtering and an increase in circuit scale.